The present invention relates to a pen for recording a selected sequence of characters according to the preamble of claim 1, as well as a method for effecting such a recording according to the preamble of claim 7.
A reader who reads a text often wishes to mark the most important sections of the text in order to be able to easily find them when next reading through the text, thereby avoiding having to read irrelevant parts of the text. It is known to use so-called highlighter pens for this purpose.
A reader may also need to put together selected parts of a text into a new document, which can be edited in a word-processing program in a computer. A known method for feeding text into a computer is to use a scanner. There are stationary scanners and portable scanners. A stationary scanner is used to scan whole pages of text, the scanner being passed over the text automatically at a constant speed. This type of scanner is not suitable for scanning selected parts of a text on a page. A portable scanner, on the other hand, can be used for this purpose.
U.S. Pat. No. 5,301,243 discloses a handheld scanner for reading characters from a character string on a substrate. The scanner, which is passed over the characters which are to be read in contact with the substrate, has an optical system which xe2x80x9cseesxe2x80x9d a small part of the substrate. The optical system comprises a CCD type line sensor, which has a plurality of light-sensitive elements arranged in a row. When the scanner is passed over the characters on the substrate, a sequence of vertical xe2x80x9cslicesxe2x80x9d of the characters and of the spaces between them is recorded. The xe2x80x9cslicesxe2x80x9d are stored in the scanner as a digital bitmap. Subsequently, OCR software (OCR=Optical Character Recognition) is used to identify the scanned characters and to store them in character-coded format, e.g. with the aid of ASCII code, in a memory. The character recognition can be performed either in the scanner or in an external computer to which the bitmapped characters have been transferred.
A problem associated with the use of portable scanners is that the scanner has to know the distance between the scanned xe2x80x9cslicesxe2x80x9d in order to be able to identify the characters correctly. This means that the scanner either has to be passed over the characters at a constant speed, which is very difficult to achieve manually, or else contain a very accurate distance meter.
In the above-mentioned U.S. Pat. No. 5,301,243, the distance problem is obviated with the aid of a mechanical distance meter in the form of a wheel. However, a mechanical distance meter comprises moving parts, which is a drawback as far as durability and manufacturing are concerned. Moreover, the distance meter makes it difficult to record information from a newspaper or the like if the latter is not placed on a hard surface. In addition, the distance meter impairs the possibility of seeing the information which is to be recorded, since the wheel has to abut against the surface from which the information is to be recorded and consequently makes the end of the scanner which abuts against the surface bulkier.
A further problem connected with known portable scanners is that they must be held orientated substantially in one and the same predetermined direction the whole time. More specifically, the sensor must be orientated perpendicular to the direction in which the pen is being moved, i.e. perpendicular to the character sequence. If a scanner is held in another direction or is rotated around its longitudinal axis while a sequence of characters is being recorded, the scanner may have difficulty identifying the characters recorded, since there is no directional information in the scanned xe2x80x9cslicesxe2x80x9d. This is a serious drawback of the prior art since different individuals often hold a scanner in different ways. In particular, there is a difference in the way right-handed and left-handed individuals hold scanners. Furthermore, an individual often changes the position of his hands as the scanner moves along a line. This is a particular problem when one is holding a book or the like in one""s hand and wishes to record text.
U.S. Pat. No. 4,949,391 discloses an optical scanner with no distance meter. More specifically, this optical scanner comprises a two-dimensional sensor, which records images with partially overlapping contents of a text over which the scanner is passed. However, the handheld scanner is really only an xe2x80x9cunintelligentxe2x80x9d recording unit which records characters without interpreting them. Instead, the OCR processing is carried out in a computer to which the scanner is connected. The scanner correlates with each other images recorded one after another in order to remove columns with redundant information from the images so that they will require less storage space. The scanner can only be moved in a predetermined direction which is defined by a wheel.
One object of the invention is to obviate, or at least reduce, the problems described above of the prior art for portable scanners, and specifically to provide a method and a pen which can be used for recording a selected sequence of characters without the aid of a distance meter and with no requirement of a constant speed of movement.
In this connection, a further object is to provide such a method and such a pen which can be held rotated in different ways around its longitudinal axis and which specifically allows the pen to be rotated around its longitudinal axis while a selected sequence of characters is being recorded, without the reliability of the identification of recorded characters being reduced in a decisive manner.
The objects are achieved by a pen and a method for recording a selected character sequence which have the features stated in claims 1 and 7, respectively.
As mentioned above, the pen and the method according to the invention are used for recording a selected sequence of characters by passing the pen over the character sequence. The sequence selected can, for example, consist of one or several words or one or several sentences selected from a longer text. The characters can be letters, numbers and or other written characters or symbols. The characters can be scanned from a paper or another medium, which can be scanned optically.
The pen comprises a light-sensitive sensor with a two-dimensional sensor surface. In this context, a two-dimensional sensor surface refers to a sensor surface which can record more than one pixel in two directions perpendicular to one another. The sensor is preferably of the CCD type. It can record two-dimensional images in grey scale or in colour. The images consist of a plurality of pixels, each being defined by an intensity value. The height of the images should be such that there is essentially room in the image for the highest character which is to be recorded. The width of the images can be chosen on the basis of the image-recording frequency desired and available sensors.
Furthermore, the pen comprises a signal processing unit which should contain at least a central processing unit, a memory, input/output circuits, and program instructions for implementation of the method according to the invention. The signal processing unit preferably comprises a suitably programmed microprocessor. The signal processing unit employs the partially overlapping contents of the images to put the recorded images together into one image. The putting-together is done both vertically and horizontally so that the vertical as well as the horizontal directional information in the images is utilised. As a result, there is no duplication of the contents in the composite image.
When an image has been put together, the signal processing unit identifies the characters in the image and stores them in character-coded format in a memory in the pen. This means that the pen is of the stand-alone type. However, it is suitably connectable to an external computer, so that the information can be transferred to the latter. A composite image preferably has a predetermined maximum length. In order to increase the speed and reduce the memory requirement for images which have been recorded but not yet put together, the putting-together is suitably carried out sequentially as the images are being recorded. Thus, the fact that the putting-together can be performed in real time reduces the memory requirement for recorded images. The putting-together normally continues until all recorded images have been put together. However, it is interrupted if the predetermined maximum length of the composite image is reached. Alternatively, the putting-together can be carried out after the recording is complete, in which case a predetermined maximum number of images car be recorded, which are subsequently put together into -an image. The pen can have an indicating device for indicating that it is ready to resume recording.
By using a sensor with a two-dimensional sensor surface, by means of which images with partially overlapping contents can be recorded, and by putting together the images into one image, it is possible to eliminate the requirement of a constant movement speed or of the provision of a distance meter. By virtue of the contents of the images partially overlapping, the images will themselves contain information about their relative position, i.e. distance information, and as a result there is no need to record this information using a distance meter. Moreover, the user can hold the pen at an optional angle of rotation in relation to the text and can even rotate the pen while recording the images without impairing the character identification, since the overlapping images contain directional information, which indicates their relative position. This considerably improves user-friendliness.
Preferably, the pen has a display, so that the user can read the recorded character sequence without first having to transfer it to an ordinary computer.
As mentioned above, in order for it to be possible to carry out the putting-together, there has to be a certain overlapping of the contents of the recorded images. The degree of overlapping should preferably be greater than 20%, preferably greater than 50%.
To make the contrast in the recorded images sufficiently great and to avoid blurriness caused by the pen being moved in relation to the character sequence, the characters which are to be recorded are preferably illuminated by light pulses when the pen is being passed over the character sequence. The illumination can suitably be achieved with the aid of illumination means, e.g. an LED, which is fitted in or to the apparatus. The frequency of the light pulses should be the same as and be synchronised with the frequency for recording images, i.e. the frequency for reading the contents of the light-sensitive sensor. A fast mechanical shutter, for example, could be used instead of the pulsed illumination. However, this results in a more complicated apparatus.
When the images are being put together, images recorded one after another are preferably examined in pairs with respect to possible overlap positions for determining the overlap position which affords the best correspondence between the contents of each pair. The overlap position determined in this manner is used when putting together the images of the pair.
When the best overlap position has been determined, a pixel intensity value is preferably determined for each overlapping pixel. This pixel intensity value is based on the intensity values of the pixel in question in the images of the pair. The pixel intensity values determined in this manner are used in the final composite image.
When the whole image has been put together, the image is preferably divided into a plurality of subimages each comprising a character. This division, which thus does not have to be a physical division but rather can be performed in the whole image, constitutes a preparation for the actual character recognition, each character preferably being interpreted separately. Alternatively, whole words or parts thereof could be interpreted.
The division of the image into sub-images can be effected by the intensity of the pixels being added up row by row and column by column in the composite image in order to obtain a plurality of row sums and a plurality of column sums and by local minima being identified among the row sums and column sums obtained.
The identification of the characters in the composite image is advantageously effected with the aid of a neural network.
Suitably, in this connection, the intensity value of each pixel in a sub-image is determined and is fed as an input signal to the neural network, which is adapted to identify the character in said sub-image with the aid of the input signals.
Prior to the identification of the characters, the composite image can be rotated if necessary so that the characters in the image form a horizontal row. This facilitates the identification.
The pen according to the invention can for example be employed to record text while studying, to scan numerical values, to compose new documents on the basis of old documents which exist in paper form only, and for many-other applications.